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AMENDMENTS TO THE SPECIFICATION: 
Please amend the Specification as follows: 

Page 1 , please replace the second full paragraph with the following amended paragraph 
An operation of taking out an individual workpiece fern from a randomly arranged stack 
of workpieces or an aggregation of workpieces contained in a container of a predetermined size, 
which have identical shapes and different three-dimensional positions/postures, teve has been 
performed manually. In storing workpieces in a pallet or placing workpieces at a predetermined 
position in a machine or a device using a (dedicated) robot, since it has been impossible to 
directly take out an individual workpiece one by one from the randomly arranged stack of 
workpieces by the dedicated robot, it has been necessary to rearrange the workpieces in 
advance so as to be picked out by the robot. In this rearrangement operation, it has been 
necessary to take out an individual workpiece from the stack manually. 

Page 2, please replace the second full paragraph with the following amended paragraph 
An image processing apparatus of the present invention compris e s includes an image 
capturing device; and a memory storing reference models created based on image data of a 
reference object captured by the image capturing device in a plurality of directions, and storing 
information of the capturing directions to be respectively associated with the reference models. 
The reference object may be an object of detection itself or an object having a shape identical to 
that of the object of detection. 

Page 2, please replace the third full paragraph with the following amended paragraph 
The image processing apparatus also compr i s e s includes a processor to perform 
matching processing on image data containing an image of the object of detection captured by 
the image capturing device with the reference models to select an image of an object matched 
with one of the reference models, and to obtain posture, or posture and position of the object 
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^tased on the selected image of the object, said one reference model and the information of the 



direction associated with said one reference model. 



Page 3, please replace the second full paragraph with the following amended paragraph 
The image capturing device may be attached to aft a wrist of aft a robot. Further, the 
image data of the reference object can be captured in a place different from a place where the 
detection of the object is performed, and supplied to the image processing apparatus on line or 
off line. 

Page 3, please replace the fo urth full paragraph with the following amended paragraph 

FIGS. 2a-2d show an e xamp le examples of reference models; 



Page 4, please replace the fourth full paragraph which continues on page 5 with the 
following amended paragraph 

An embodiment in which an image processing apparatus of the present invention is used 
in combination with aft a robot system will be described. In this embodiment, an image of a 
stack of workpieces, which are objects of detection having identical shapes and randomly 
arranged as shown in FIG. 1 , is captured by an image capturing device (camera or visual 
sensor) 20, which is attached to a wrist of a robot RB and position and posture (orientation) of 
^ the individual workpieces are detected based on the captured image. For this purpose, images 
of a reference object, which is one of workpieces W subjected to a picking operation or an object 
having a shape identical to that of the workpiece W are captured in different directions by the 
image capturing device and reference models are created from the image data obtained by the 
image capturing and stored in advance. Matching processing between the image data obtained 
by capturing the image of the stack of workpieces and the reference models is executed to 
select an image of one workpiece matched with one of the reference models, and a 
position/posture of the selected workpiece is determined based on the selected image of the 
workpiece in the image field of view, the selected one of taught modes and the position/posture 
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Information associated with the selected one of the reference nnodels. 



Page 5, please replace the third full paragraph which continues on page 6 with the 
following amended paragraph 

L ; 

The servo control section 5 compr i s e s includes servo controllers 5a1 to San (n: sum of 
the number of all the axes of the robot including additional movable axes of a tool attached to a 
wrist of the robot), each composed of a processor, a ROM, a RAM, etc. Each servo controller 
performs position/velocity loop control and also current loop control for its associated servomotor 
for driving the axis, to function as a co-called digital servo controller for performing loop control 
of position, velocity and current by software. Each servomotor M1-Mn for driving each axis is 
drivingly controlled according to outputs of the associated servo controller 5al-5an through the 
associated servo amplifier 5b1-5bn. Though not shown in FIG. 3, a position/velocity detector is 
attached to each servomotor M1-Mn, and the position and velocity of each servomotor detected 
by the associated position/velocity detector is fed back to the associated servo controller Sal- 
Sam. To the input-output interface 6 are connected are sensors of the robot, and actuators and 
sensors of peripheral devices. 

Page 6, please replace the first full paragraph with the following amended paragraph 
FIG 4 is a block diagram of the image processing apparatus 30 connected to aft the 
interface 7 of the robot controller 1 0. The image processing apparatus 30 compr i s e s includes a 
processor 31 to which a ROM 32 for storing a system program to be executed by the processor 
31 , an image processor 33, an image-capturing-device interface 34 connected to the image 
capturing device 20, a an MDI 3S with a display such as a CRT or a liquid crystal display for 
inputting and outputting various commands and data, a frame memory 36, a nonvolatile memory 
37, a RAM 38 for temporary storage of data and a communication interface 39 for the robot 
controller are connected. An image captured by the camera 20 is stored in the frame memory 
36. The image processor 33 performs image processing of images stored in the frame memory 
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36 on demand of the processor 31 so as to recognize an object. The architecture and function 
of the image processing apparatus 30 itself is no way different fefm from the conventional image 
processing apparatus. The image processing apparatus 30 of the present invention is different 
fem from the conventional one in that reference models as described later are stored in the 
nonvolatile memory 37 and pattern matching processing is performed on an image of a stack of 
workpieces W captured by the image capturing device 20 using the reference models to obtain a 
position and posture of a workpiece W. 

Page 6, please replace the second full paragraph which continues on page 7 with the 
following amended paragraph 

The image capturing device 20 is used for obtaining image data, as described later, and 
may be a CCD camera for obtaining two-dimensional images data or a visual sensor capable of 
obtaining three-dimensional image data including distance data. In the case of using the CCD 
camera, the image data is obtained by a conventional method based on two-dimensional images 
captured by the CCD camera, but in cases of the visual sensor capable of obtaining three- 
dimensional data including distance data, two-dimensional arrangement data with distance data 
between the sensor and an object is obtained. The visual sensor for obtaining the three- 
dimensional data including distance data is known, for example, from a three-dimensional visual 
s e nsors sensor of a spot light scanning type disclosed in Japanese Patent Publication No. 7- 
270137, and the summary to the three-dimensional visual sensor is described below. 

Page 7, pl6ase replace the first full paragraph with the following amended paragraph 

This visual sensor detects a three-dimensional position of an object by irradiating a light 
beam to form a light spot on the object for scanning the object in two differerit directions (X 
direction and Y direction) and by detecting the light reflected on the object by a position sensitive 
detector (PSD). Three dimensional position of the object is measured by a calculation using the 
respective inclination angles 9x, 6y of mirrors for scanning and aft incident positions of the 
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reflected light beam on the PSD. 

Page 7, please replace the third full paragraph which continues onto page 8 with the 
following amended paragraph 

Scanning range (measuring range) on an object is set in advance, and an inclination 
angle 9x, 9y of the mirrors is controlled discretely. As shown in FIG. 7, the scanning is 
performed from a point (1, 1) to a point (1, n), from a point (2,1) to a point (2, n), .... from a point 
(m, 1) to a point (m, n) on the X-Y plane within the scanning range, to measure three- 
dimensional positions of each reflected point on the object. Also, a distance Z (i, j) between the 
sensor and the reflection point (i, j) on the object is obtained and stored in the RAM38 RAM 38 
of the image processing apparatus 30. Thus, the image data is obtained as two dimensional 
arrangement data including the distance data Z (i, j) between the sensor and the reflection point 

on the object, as shown in FIG. 8, 

^ — ■ ^ — 

Page 8, please replace the third full paragraph which continues onto page 9 with the 
following amended paragraph 

Then the index i is incrementally increased by "1" and the inclination angle ex of the 
mirror for X-axis direction scanning is increased by the predetermined amount Ax (Step 306, 
307). It is determined whether or not the index i exceeds the set value n (Step 308). If the index 
i does not exceed the set value n, the procedure returns to Step 303 and the processing from 
Step 303 to Step 308 is executed to obtain the distance Z (i, j) of the next point. Subsequently, 
the processing of Steps 303-308 afe is reportedly executed until the index i exceeds the set 
value n to obtain and store the distance Z (i, j) of the respective points (1 , 1 ) to (1 , n) shown in 
FIG. 7. 

Page 9, please replace the first full paragraph with the following amended paragraph 
If it is determined that the index i exceeds the set value n in Step 308, the index i is set to 
"1" and the index j is incrementally increased by "1" to increase the inclination angle 9y of the 
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mirror for Y-axis direction scanning (Steps 309-311). Then, it is determined whether or not the 
index] exceeds the set value m (Step 312) and if the index j does not exceed the set value m, 
the procedure returns to Step 302 to repeatedly e x e cut e s execute the processing of Step 302 
and the subsequent Steps. 

Page 9, please replace the second full paragraph with the following amended paragraph 

Thus, the processing from Step 302 to Step 312 is repeatedly executed until the index j 
exceeds the set value m. If the index j exceeds the set value m, the points in the measurement 
range (scanning range) shown in FIG. 7 jp^ have been measured entirely, the distance data Z 
(1 , 1 ) - Z (m, n) as two-dimensional arrangement data are stored in the RAM28 RAM 28 and the 
image data obtaining processing is terminated. A part of the image data of two-dimensional 
arrangements or a plurality of distance data can be obtained by appropriately omitting the 
measurement of the distance for the i ndex i. 

Page 11, please replace the fourth full paragraph which continues onto page 12 with the 
following amended paragraph 

As described above, the 0-th position/posture of the robot at which the camera 20 
captures the image of the object, and the rotation axis and the rotation angels with respect to the 
0-th position/posture are set in advance in order to determine the subsequent positions/postures 
of the reference workpiece, and also the number of the subsequent positions/postures of the 
workpiece are set. For intelligible explanation, it is assumed that an optical axis of the camera is 
parallel to the Y-axis of the world coordinate system and that a position where the X-axis and Y- 
axis coordinate values are identical to those of the reference workpiece and only the Z-axis 
coordinate value is different fefm from that of the position of the reference workpiece is taught to 
the robot as the 0-th image capturing position for obtaining the 0-th reference model. Further, 
the positions of the robot where the camera is rotated with respect to the reference workpiece by 
30 degrees, 60 degrees and 90 degrees along the axis passing the central point of the reference 
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workpiece and parallel to the X-axis of the world coordinate system are set as the 1st, 2nd and 
3rd image capturing position, and the number N of the image capturing positions is set "4." 

Page 12, please replace the first full paragraph which continues onto page 13 with the 
following amended paragraph 

When a command of creating reference models is inputted from teaching operation 
panel 4, the processor 1 of the robot controller 10 sets a counter M for counting the number of 
the image capturing to "0" (Step 100). The robot is operated to have the M-th position/posture 
and a command for image capturing is outputted to the image processing apparatus 30 (Step 
101). In response to this command, the image processing apparatus 30 performs capturing of 
an image of the reference workpiece with the camera 20 and the captured image data is stored 
in the frame memory 36 (Step 102) . Further, relative position/posture of the workpiece with 
respect to the camera is obtained and stored in the nonvolatile memory 37 as relative 
position/posture of M-th reference model, and a data-captured signal is sent to a robot controller 
(Step 103). Thus, position/posture of the workpiece is a camera coordinate system set to the 
camera is obtained from the position/posture of the camera and the position/posture of the 
reference workpiece in the world coordinate system when capturing the image by the camera, 
and is stored as the relative position/posture of the workpiece with respect to the camera. For 
example, the position/posture of the workpiece in the camera coordinate system is stored as [xO, 
yO, zO, aO, pO, yO]c, where a, p and y mean rotation angle around X-,Y-. Z- axes, and "c" means 
the camera coordinate system. 

Page 13, please replace the first full paragraph with the following amended paragraph 

Upon receipt of the data-captured signal, the processor 1 of the robot controller 10 
incrementally i ncr e as e increases the value of the counter M by "1"(St e p 10 4 ) "1" (Step 104) and 
dotorm i n e determines whether or not the value of the counter M is less than a set value N (=4) 
(Step 105). If the value of the counter M is less than the set value N, the procedure returns to 
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Step 101 to move the robot to the M-th innage-capturing position/posture. Thus, in the example 
as shown in FIGS. 2a-2d, the camera is successively turned by 30 degrees around the axis 
parallel to X axis of the world coordinate system and passing the workpiece position, and 
successively captures the image of the workpiece, and reference models and relativity 
positions/postures of the camera with respect to the workpiece at the image capturing are 
stored. 

Page 1 3, please replace the second full paragraph with the following amended 
paragraph 

Processing of Steps 101-105 afe js repeatedly executed until the value of the counter M 
equals to the set value N (=4), and the reference models and the relative positions/postures of 
the camera and the workpiece are stored in the nonvolatile memory 37. Thus, the reference 
models created from the image data of the workpiece at the positions/postures shown in FIGS. 
2a-2d are stored, and the relative positions/postures between the camera and the workpiece for 
respective reference models are stored as positions/postures of the workpiece W in the camera 
coordinate systems as [xO, yO, zO, aO, (30, yO]c, [x1, y1, zI, a1, pi. y^Jc, [x2, y2, z2, a2, P2, y2]c, 

and [x3, y3, z3, a3, (33, y3]c. 

Page 14. please replace the fourth full paragraph which continues onto page 15 with the 
following amended paragraph 

FIG. 6 is a flowchart of processing for the picking operation. When a picking command 
is inputted into the robot controller 10 from the teaching operation panel 4, the processor 1 
operates the robot RB to move the camera attached to the robot wrist to an image capturing 
(^^^ position where a stack of workpieces afe is within a field of view of the camera 20 (Step 200). 
There-dimensional position/posture of the camera 20 on the world coordinate system at this 
image capturing position is outputted to the image processing apparatus 30, and a image 
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capturing command is outputted (Step 201 ). Upon receipt of the image capturing command, the 
processor 31 of the image processing apparatus 30 captures an image of the stack of the 
workpieces W, to obtain image data of some workpieces W and store it in the frame memory 36 
(Step 202). 

Page 15, please replace the first full paragraph with the following amended paragraph 
Then, pattern matching processing is performed for the image data stored in the frame 
memory 36 using one of reference models (1st reference model) stored in the nonvolatile 
memory 37 so as to detect a workpiece W (Step 203). In this pattern matching processing, 
matching of the image data of the reference model with the image data of workpieces is 
performed on the basis of position, turn and scale. It is determined whether or not an object 
hav i ng has a matching value equal or greater than the set value (Step 204). If an object having 
a matching value equal or greater than the set value is not detected, the procedure proceeds to 
Step 205 to determine whether or not the pattern matching is performed using all the reference 
models (1st to 4th reference models). If the pattern matching using all of the reference models 
is not yet performed, further pattern matching is performed using another reference model (Step 
206). 

Page 15, please replace the second full paragraph which continues onto page 16 with 
the following amended paragraph 

If it is determined in Step 204 that an object having a matching value equal or greater 
than the set value with respect to any of the reference models is detected, the procedure 
proceeds to Step 207 to perform matching processing on the two-dimensional data of the 
detected workpieces W, using every taught mode. In Step 208, the reference model having the 
most l arg e highest matching value in the pattern matching processing is selected, and the 
relative position/posture of the workpiece W with respect to the camera 20 is determined based 
on the relative position/posture of the camera and the reference workpiece stored for the 
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selected reference model, and position, rotation angle and scale of the image of the workpiece 
in the matching processing, (Step 208). The position and posture (orientation) of the detected 
workpiece on the world coordinate system is determined from the position and posture of the 
camera 20 in the world coordinate system, which has been sef^ set in Step 201, and the relative 
position/posture of the workpiece W with respect to the camera 20, and is outputted (Step 209). 
Thus, since the relative position/posture of the workpiece W with respect to the camera 20 is the 
position/posture of the workpiece W in the camera coordinate system, the position and posture 
(orientation) of the detected workpiece W in the world coordinate system is obtained by an 
arithmetic operation of coordinate transformation using the data of the position/posture of the 
workpiece W in the camera coordinate system and the position/posture of the camera 20 in the 

worid coordinate system (Step 209). 

Page 17, please replace the fourth full paragraph with the following amended paragraph 
Further, in the case where a wide-angle lens is installed in the CCD camera as the image 
capturing device, for example, there is possibility of judging the inclination angle to be 30 
degrees by influence of parallax when a workpiece of zero degree inclination is at a corner of a 
field of view of the camera. In such a case, the camera may be moved para llell y parallel in 
accordance with the position of the workpiece in the field of view of the camera to a position right 
above the workpiece to lose influence of parallax, and at this position the image capturing 
processing of Step 201 and the subsequent Steps in FIG. 6 is are performed so that the false 
judgment is prevented. 
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